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ABSTRACT 

Enlarging upon the scope and agenda of earlier 
diagnostic research, a study was conducted to examine (1) teacher 
responses to receiving diagnostic information about some of their 
students, (2) effects on student achievement of teachers receiving 
both diagnostic information and diagnostic training, (3) diagnostic 
classifications that emerged from student performance, and (4) 
differential teacher effectiveness and instructional practices. 
Subjects included 10 experienced fifth-grade classroom teachers, A 
reading diagnostic battery, developed for the previous studies, was 
individually administered as a pretest to 186 of the 192 students in 
the teachers' classrooms. An experimental group of five teachars, 
randomly assigned, received four hours of training in diagnostic 
reliability. Before and after the training, all 10 teachers diagnosed 
one of four randomly assigned simulated cases of reading difficulty 
at two different times. Teachers were also interviewed three times. 
Results showed that teachers responded positively to the receipt of 
specific diagnostic information about their students, seeing it as 
useful for thinking about and making changes in their instructional 
practices. Students for whom teachers received diagnostic information 
did not show significantly improved achievement over students for 
whom no information was provided. (Appendixes include the reading 
diagnostic battery and the interview questions.) (HOD) 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*************************************** ******************************** 



ERLC 



U A DEPARTMENT Of EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 
EDUCATIONAL RESC JRCES INFORMATION 
/ CENTER <E«Cr 

fiJThfc document has bevn r*productd « 



Ttc#rv*d from tt* ptrron or ofganufiDon 
ongmrtmg it. 
O Minor changes havt bmn m*d. to tmprovt 
reproduction quality. 

• Ponti of v>tw or opinion* rated m tho cJocu- 
mtnt do not ntctssanty reprtcwtt officii N!E 
fxvshion or poUcy 



Research Series No. 162 



USING STUDENT DIAGNOSTIC INFORMATION 
TO ESTABLISH AN EMPIRICAL DATA BASE IN READING 

Annette B. Weinshank, Ruth M. Polin, 
and Christian C . Wagner 



The Institute for Research on Teaching 
252 Erickson Hall 
Michigan State University 
East Lansing, Michigan 48824 




Published By 



June 1985 



This work is sponsored in part by the Institute for Research on Teaching, 
College of Education, Michigan State University. The Institute for Research 
on Teaching is funded primarily by the Program for Teaching and Instruction of 
the National Institute of Education, United States Department of Educatior. 
The opinions expressed in this publication do not necessarily reflect the 
position, f licy, or endorsement of the National Institute of Education. 
(Contract No. 400-81-0014) 



BEST COPY AVAILABLE 



Institute for Research on Teaching 



The Institute for Research on Teaching was founded at Michigan State 
University in 1976 by the National Institute of Educa tion F °"°^ in 8 a 
nationwide competition in 1981, the NIE awardeu a second c ° n t x . r ^ agencies S 
extending work through 198*. Funding is also received from other agencies ana 
foundations for individual research projects. 

The IRT conducts major research projectr aimed at improving classroom 
teaching, including studies of classroom management «»?aies, araal- 
ization, the diagnosis and remediation of reading difficulties, and teacher 
location IRT researchers are also examining the teaching of specific school 
subjects such as reading, writing, general mathematics, an d sciavoe, and are 
seeking to understand how factors outside the classroom affect teacher decision 
making. 

Researchers from such diverse disciplines as educational psychology, 
anthropology, sociology, and philosophy cooperate in conducting IRT ^search. 
They join forces with public school teachers, who work at the IRT as half-time 
collaborators in research, helping to design and plan studies, collect data, 
analyze and interpret results, and disseminate findings. 

The IRT publishes research reports, occasional papers, conference pro- 
ceedings? a newsletter for practitioners, and lists and catalogs of IRT .pubUca- 
tfons. For more information, to receive a list or catalog, and/or to be placed Ion 
the IRT mailing list to receive the newsletter, please write to the IRT Editor, 
{nstitute^or Research on Teaching, 252 Erickson Hall, Michigan State Univer- 
sity, East Lansing, Michigan *882f-103*. 



Co-Directors: Here E. Brophy and Andrew C. Porter 
Associate Directors: Hudith E. Lanier and Richard S. Prawat 

Editorial Staff 
Editor: Janet Eaton 
Assistant Editor: Patricia Nischan 



9 

ERIC 



3 



Abstract 

This study is the last in a programmatic series of research studies that 
began by investigating liagnostic reliability in reading, continued by de- 
veloping a training program for improving diagnostic reliability, and con- 
cluded by investigating how diagnostic information might be used to improve 
student achievement in reading. Results show that 9 of the 10 teachers in the 
final study felt that receipt of diagnostic information sharpened their obser- 
vations of student reading behavior and had potential for directing their 
thinking about instructional changes in their respective reading programs. 

Students for whom teachers received diagnostic information did not show 
significantly improved achievement over students for whom no information was 
provided. Eighteen diagnostic classifications emerged from student perfor- 
mance on reading diagnostic tests; five classifications accounted for 60% of 
the students in the study. The characteristics of these five classifications 
together with their instructional implications are presented. Teachers were 
found to be differentially effective (p less than .05) for word recognition, 
oral reading, listening comprehension, and silent reading comprehension 
outcomes. Available interview and observation data were not sufficient for 
specifying instructional practices associated with varying levels of student 
achievement. The system documented by the study is a means of establishing 
an empirical basis for improving instruction and outcomes in reading. 
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USING STUDENT DIAGNOSTIC INFORMATION 
TO ESTABLISH AN EMPIRICAL DATA BASE 
IN READING 

Annette B. Weinshank, Ruth M. Polin, and Christian C. Wagner* 

In the teaching of reading, the absence of specific information linking 
student outcomes back to reliable diagnosis and instruction has presented a 
major obstacle to practitioners' learning systematically from their own ex- 
perience. Improving student outcomes in reading requires that the teacher 
receive information that reliably classifies student performance into appro- 
priate diagnostic categories. If the diagnostic categorizations are unreli- 
able, it is not possible to determine, when outcomes are poor, whether 
instruction was inadequate for the student's problem or whether student per- 
formance was incorrectly diagnosed and good instruction was wasted on the 
wrong problem. 

Unfortunately, diagnostic unreliability has been shown to be a pervasive 
problem in the field of reading. Only recently has it become possible to 
remedy the situation and to provide teachers with reliable diagnostic informa- 
tion. A series of studies on the diagnostic reliability of educational prac- 
titioners (Weinshank, 1982; Vinsonhaler, Weinshank, Wagner, & Polin, 1983; 
Weinshank & Vinsonhaler, 1983) showed that reading and learning disability 
specialists and classroom teachers did not agree with themselves or with one 
another in their diagnostic judgments about simulated cases of children with 
reading problems. 



L Annette B. Weinshank is an IRT teacher collaborator and coordinator of 
the just completed Outcomes in Reading Project. Ruth M. Polin is a research 
associate with the same project. Christian C. Wagner, who teaches in the 
College of Engineering at Oakland University, served as the project's 
consultant. 

The authors would like to acknowledge, with thanks, Lynn Wilhelm's 
contributions during the data-collection phase of this research. 
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A second series of studies (Vinsonhaler , Weinshank, Polin, & Wagner, 
1983) showed that the reliability of diagnostic decision making could be im- 
proved drams cically through the use of a specific type of training program. 
T„o important products of the training studies were (l) a reading diagnostic 
battery based primarily on a model of reading that focuses on performance in 
work recognition, oral reading, silent reading comprehension, and listening 
comprehension and (2) a computer program for generating reliable diagnoses 
given the data from student performance on the battery. Thus it became pos- 
sible to provide teachers with reliable diagnostic information about their 
students' reading performances. 

in a final set of studies (Wagner, 1982; Weinshank, Polin, & Wagner, 
1985), one junior high school remedial reading teacher was provided with reli- 
able diagnostic information about her students' reading performance. The 
studies were designed to test the methods needed to move from diagnostic reli- 
ability to issues of diagnostic validity: verifying linkages that exist 
between diagnosis, instruction, and student achievement. That is, given reli- 
able diagnostic information, what instructional plans would be carried out and 
with what effect on students? These studies were limited in two ways: They 
involved only one teacher and they dealt with a restricted range of student 
abilities. 

The study reported here enlarged the scope and agenda of the earlier work 
in order to answer four major research questions. 

1. What do teachers report about the utility of "^^S?"* 
diagnostic and performance information about their students/ 

information? 
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3. What are the diagnostic classifications that emerge from student 
performance on the test batteries? 

4. Are teachers differentially effective for students in different 
diagnostic classifications? What teacher practices are associated 
with gains in achievement in particular diagnostic classifications? 

Procedures 

The study was conducted in an urban school district in mid-Michigan. All 
fifth-grade teachers in the district were invited to participate; those who 
volunteered were paid a small honorarium. Ten experienced classroom teachers 
ultimately joined the stud/. Eight of the 10 classes were heterogeneous in 
composition, with 19 to 28 students per class. The 9th class (with 6 stu- 
dents) was a remedial one; the 10th (with 7 students) an enriched one. 

The reading diagnostic battery (Appendix A), developed for the previous 
studies, was individually administered as a pretest to 186 of the 192 students 
in the teachers' classrooms. Trained testers (interrater reliability ■ 92 zo 
99%) admiaistered the battery and recorded (1) student responses to a set of 
interview questions and (2) student performance on four diagnostic reading 
behaviors: word recognition (correct, incorrect, and evidence of decoding 
strategies), oral reading (errors, rate, fluency, and inflection), silent 
reading comprehension (rate and percent of paragraph recalled), and listening 
comprehension (percent of paragraph recalled). 

Teachers were interviewed three times. During the first interview 
(Appendix B), data was obtained about their teaching practices during the 
daily formal reading session (instructional practices and materials, classroom 
organization, criteria for grouping, nature of feedback, etc.). Tsachers were 
then observed once during the formal reading period in order to verify whether 
the interview data did, in fact, reflect their classroom practices. 
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After the first interview, pretest information was returned to each 
teacher. The information consisted of individual scores and reading group 
mean scores for each of the four diagnostic reading behaviors. Tables 1 and 2 
show the Information that one teacher received about her students' performance 
on word recognition. The individual data were provided for two-thirds of the 
students in each class, randomly selected. Pretest information was given to 
the teachers together with a letter explaining how to read the data; no in- 
terpretation of data was made. The remaining one-third of the students served 
as an internal control group for the hypothesis that teachers' receipt of 
diagnostic information has an effect on student achievement. The data for 
these students were not given to the teacher. 

Table 1 

Pretest Word Recognition Scores by Individuals— Teacher C 



Grade Level 



Percent cf words decoded 
0-3 3.1-6 
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8.5 - 9.0 5C02* 

8.0 - 8.4 5C04 5C08 

5C09 5C14 

7.5 - 7.9 5C01 

7<0 _ 7.4 5C06 5C11 5C12 

6.5 - 6.9 5C15 

6.0 - 6.4 5C05 

5.5 - 5.9 

5.0 - 5.4 

4.5 - 4.9 5C03 

4.0 - 4.4 5C17 

Control students not Included; 5C04. 5C13, 5C19, 5C07 r 5C16, 5C10 

a This is the code for a student's name. 
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Table 2 



Pretest Word Identification Scores by Reading Group—Teacher C 





Weavers(n-5) 
(H/M) 


Gateways(n**5) 
(H/M) 


Barcne re (n=7) 
(H/M) 


Word recognition and analysis 








Slosson Oral Reading Test (SORT) 








1c grade equivalent & range 


6.7 (4.6-8.0) 


6.7 (A. 3-8.1) 


7.7 (6.9-8.5) 


x percent decoding & range 


5.0 (1 - 11) 


2.0 (0 - 5) 


4.0 (0 - 9) 



x = mean (average) 

H/M - Houghton-Mifflin Reading Series 

Weavers, Gateways, and Banners are basal texts for grades, 3, 4, and 5, 
respective ly. 



An experimental group of five teachers, randomly assigned, received four 
hours of training in diagnostic reliability. We thought that such training 
would enable them to better interpret their students' performance scores, ul- 
timately resulting in changes in instruction and improved achievement. Before 
and after the training, all 10 teachers diagnosed one of four randomly as- 
signed simulated cases of reading difficulty at two different times. Each 
case presented a commonly encountered problem of reading difficulty. Case 
material included completed test booklets, test scores, background informa- 
tion, and the like (Lee & Weinshank, 1978). The second diagnosis was done on 
an alternate version of the case used for the first diagnosis. The purpose 
of the diagnoses was to control for non-experimental sources of gain. 
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Teachers were interviewed a second time (Appendix C) to determine, among 
other things, (a) how they evaluated the pretest information they received and 
(b) what influences *ie simulated case diagnoses (and training program, where 
applicable) had on their interpretation of their students' pretest results. 

Of the 186 students who had been pretested, 169 remained by the end of 
the school year. These students were given the diagnostic battery again as a 
posttest. Data for posttest and achievement were analyzed for both indi- 
viduals and reading groups and the resulting information was returned to each 
teacher shortly after the 1983-84 school year ended. Scores for all the 
students in the class, including control students, were provided. Tables 3 
and 4 exemplify the information returned to the teachers. 

Table 3 



Word Recognition Achievement by Individuals — Teacher C 



Decreases Increases 


Grade 
Level 
Changes 


Changes in Percent of Decoding 


More 
than 
15% 


11-15% 


Decreases 
6-10% 


0-5% 


Increi 
0-5% 


ises 
6-10% 


1.6-2.0 














1.1-1.5 








5C05 


5C17 




.6-1.0 








5C11 
5C13 
5C18 


5C01 




0-.5 






5C07 


5C09 5C14 
5C10 5C16 
5C12 5C19 


5C03 
5C08 




.1-.5 






5C06 








.6-1.0 
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Table 4 

Achievement by Reading Group--Teacher C 





Weavevs(n=5) 


Gateway s(n=5) 




(H/M) 


(H/M) 


Word recognition and analysis 






Slosson Oral Reading Test (SORT) 






x grade equivalent changes & range 


+.3 (0 to .5) 


+.8 (.2 to 1.3) 


x percent decoding change s& range 


-2.5 (-8 to 3.5) 


-1.3 (-5 to +2) 



Banners (n«6) 

£h7m) 



+ .4 (-.2 to ,6) 
-2.8 (-7 to +2) 



x « mean (average) 

H/K « Houghton-Mifflin Reading Series 



The final teacher interview (Appendix D) focused on whether the post test 
and achievement scores for 1983-84 had any effect on the participating 
teachers 1 instructional decisions and actions for the 1984-85 school year. 

Results 

What do teachers report about the utility of receiving reliable diagnostic 
and performance information about their students? 

After receiving the pretest diagnostic information, teachers were asked 
about its usefulness to them. During this second interview, 7 of the 10 
teachers reported that the information was useful in sharpening their observa- 
tions of student reading behavior in a variety of ways. The information (1) 
caused them to expect more from the studerts who scored well, (2) prompted 
them to recheck student group placement, (3) directed their attention to 
student shortcomings, and (4) verified their intuition about student group 
placement. The remaining three teachers felt the data were not applicable for 
use in their classrooms. 
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Teacher responses about the usefulness of the posttest and eichievement 
information were recorded in the third interview, which was conducted at the 
beginning of the following school year. This time, 9 of the 10 teachers re- 
ported that the information had potential for directing their tiinking about 
instructional change in the reading program. However, they cited a number of 
limitations that would have to be corrected before the information could be 
put to direct use. 

They wanted to (1) receive the information before the school year ended, 
(2) discuss their students 1 test results with the research staff and/ or the 
other participating teachers, and (3) receive instructional strategies tar- 
geted to the diagnostic test results. 

What are the main effects on student achievement of whether or not a teacher 
receives (a) reliable diagnostic and performance information about a 
particular student and (b) diagnostic training that uses this type of 
information? 

There were no statistically significant differences in achievement for 
the four major diagnostic categories (word recognition, oral reading, silent 
reading comprehension, and listening comprehension) for experimental students 
whose diagnostic test results were returned to the teachers versus those for 
whom no diagnostic information was provided. It may be that teachers need to 
go through an entire year of receiving and using diagnostic information before 
tliey are able to incorporate use of it into their daily practice. 

Similarly, there were no statistically significant differences in 
achievement for the students of teachers who received four hours of diagnostic 
training versus the students of the untrained teachers. Additionally, the 
training did not measurably improve the diagnostic reliability of the teachers 
on the simulated cases. While the four-hour training program was apparently 
too brief to alter teacher reliability, it was useful to the teachers in their 
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made was whether performance on one factor was above or below performance on 
the other. 

Dividing a dimension into only two parts is probably too coarse a dis- 
tinction, It may be the case that students separated by quarter of a year or 
even less require differing instruction. It is not known, a priori, how fine 
the performance distinctions should be. Choosing fewer distinctions rather 
than more can be justified on the grounds of practical as well as research 
constraints. As a practical matter, teachers cannot accommodate a differenti- 
ating scheme that results in an excessive number of instructional groups. As 
a research issue, making very fine distinctions among students would require a 
large sample to insure that there would be enough students falling into any 
one diagnostic category under study. Therefore, both the limited sample size 
of this study as well as the necessity that the diagnostic information be use- 
ful to the teachers dictated that fewer rather than more distinctions be used 
with each dimension. 

Given the dimensions of interest, six diagnostic possibilities were 
formulated: 

1. Word recognition below 4*5 grade equivalent (indicates problem) 
or above 4.5 grade equivalent (indicates no problem). 

2. Oral reading below 4.5 grade equivalent (problem) or above 4.5 
or above 4.5 grade equivalent (no problem). 

3. Oral reading in relation to word recognition: word recognition 
grade equivalent is either above or below oral reading grade 
equivalent* Consider the following example- 

a. Oral reading and word recognition performance are both at 
grade placement. In this case, better oral reading 
performance is no being limited by word recognition 
ability. 

b. Oral reading is at grade placement but word recognition 
performance is below grade placement. This diagnostic 
classification represents a potential problem m oral 
reading for the student. 
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c. Oral reading is below grade placement but word recognition is 
adequate. The teacher must look to other limiting causes for the lag 
in oral reading. 

d. Oral reading and word recognition are both inadequate. Oral reading 
is limited, at the least, by word recognition deficits. 

4. Silent reading comprehension i3 below 4.5 grade equivalent (problem) or 
above 4.5 grade equivalent (no problem), 

5. Silent reading comprehension in relation to word recognition: Word 
recognition is either above or below 3ilent reading comprehension. 

6. Silent reading comprehension in relation to listening comprehension: 
Listening comprehension is either above or below silent reading comprehen- 
sion. (Note that listening comprehension is not included as a separate 
dimension. Its primary relevance is as an indicator of silent reading 
comprehension potential.) 

Given these six dimensions together with the two values per dimension, a 

total of 64 or 2^ diagnostic classifications, were generated. All the 

students in the study fell into only 18 of the 84 classifications. Of these 

18, 5 accounted for 60% of the students. No students fell into any of the 

remaining 46 classifications. Following are the 5 major classifications. 

Classification 1: N = 16 
Diagnosis: 

word recognition: no problem 

oral reading: problem 

oral reading below word recognition 

silent reading comprehension: problem 

silent reading comprehension below word recognition 

silent reading comprehension below listening comprehension 

One could speculate that students in this classification have learned the 

mechanics of decoding but have not generalized those skills to extended text 

reading, either orally or silently. Given the relationship between silent 

reading comprehension and listening comprehension, these students would seem 

to have the potential for satisfactory reading performance. 

Classification 2: N = 25 
Diagnosis: 

word recognition: no problem 

oral reading: no problem 

oral reading below word recognition 

ERIC 14 
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silent reading comprehension; problem 

silent reading comprehension below word recognition 

silent reading comprehension below listening comprehension 

This classification is the same as Classification 1 except that oral 
reading is not a problem. These students may be reading orally on a super- 
ficially satisfactory basis but have not developed the monitoring and tracking 
strategies required for comprehension. 

Classification 3; » - 24 
Diagnosis: 

word recognition: no problem 

oral reading: no problem 

oral reading below word recognition 

silent reading comprehension: no problem 

silent reading comprehension below word recognition 

silent reading comprehension at or above listening comprehension 

These students are adequate fifth-grade readers. Since both their oral 

reading and silent reading comprehension are below their word recognition 

performance, these students might be expected to perform orally and silently 

at a higher level. These students may need instruction that enlarges their 

vocabulary, conceptual base, and the like. 

Classification 4: N =* 19 
Diagnosis: 

word recognition: no problem 

oral reading: no problem 

oral reading at or above word recognition 

silent reading comprehension: problem 

silent reading comprehension below word recognition 

silent reading comprehension below listening comprehension 

These students* word recognition and oral reading performances are 

similar. Therefore, as word recognition improves, oral reading ability could 

be expected to rise correspondingly. Because silent reading comprehension is 

below word recognition and listening comprehension performance, other factors 

must be found to explain the silent reading comprehension problem. 
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Classification 5: N - 16 
Diagnosis: 

word recognition: no problem 

oral reading: no problem 

oral reading at or above word recognition 

silent reading comprehension: no problem 

silent reading comprehension below word recognition 

silent reading comprehension at or above listening comprehension 

These students appear to be performing up to their current limits. 

Diagnostic classifications and grouping practices . The teachers 9 
grouping practices were examined in relation to the diagnostic classifications 
generated for their students. Table 5 shows the profiles for the students in 
a classroom representative of the six who had two or more reading groups. 

Table 5 

Teachers 1 Groups in Relation to Student Diagnostic Classifications 



Low Group Mid Group High Group 

WR OR SRC WR OR SRC WR OR SRC 



NP NP NP 

NP NP NP 

NP P P 

NP NP P 

NP NP P 



NP NP P 

NP NP NP 

NP NP NP 

NP NP P 

P P P 



NP NP NP 

NP NP NP 

NP NP NP 

NP NP P 

NP NP NP 

NP NP NP 

NP NP P 



WR « Word Recognition 

OR « Oral Reading 

SRC 3 Silent Reading Comprehens 



P = Problem: below A. 5 grade level 
NP = No Problem: at 4.5 or above 
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As can be seen, the diagnostic testing showed that most students had ade- 
quate pretest performance; the three groups were virtually indistinguishable 
from one another. To the extent that the groups differed noticeably, they did 
so on the comprehension dimension. Even here, not all students in the low 
group were deficient and not all students in the high group were proficient. 
Overall, ouly two students were deficient in more than one classification,, 
The lack of clear-cut differences among reading groups was observed in all 
classes where there were two or more groups. This leads to questions about 
whether the differences that did exist warranted multiple groupings and the 
resulting use of different levels of the basal text. 

Are teachers differentially effective for students in different diagnostic 
classifications? What teacher practices are associated with gains in 
achievement in particular diagnostic classifications? 

Table 6 summarizes the performance and outcomes for word recognition, 

oral reading, silent reading comprehension, and listening comprehension for 

all the students in the study. 



Table 6 

Mean Performance and Achievement Scores Across All Students 





Pretest (GE) 


Posttest 


(GE) 


Outcomes (in years) 


WR 


6.4 (2.4 - 9.4) 


7.0 (2.6 - 


9.6) 


.51 (-0.5 to 1.7) 


OR 


5.7 (1.0 - 9.0) 


6.3 (1.0 - 


9.0) 


.56 (-2.0 to 4.0) 


SRC 


4.3 (3.0 - 8.0) 


4.7 (3.0 - 


8.0) 


.38 (-2.0 to 4.0) 


LC 


4.9 (2.0 - 8.0) 


5.2 (3.0 - 


8.0) 


.28 (-3.0 to 4.0) 




GE ■ Gra 'e Equivalent 
WR a Worci Recognition 
SRC » Silent Reading Comprehension 


OR = 
LC - 


Oral Reading 

Listening Comprehension 
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As the table showSj these students entered fifth grade, on the average, 
above grade level in word recognition and oral reading, at grade level in 
listening comprehension, and below grade level in silent reading comprehen- 
sion. While the mean gains in word recognition and oral reading were only 
half a year for one year in school, the students still entered sixth grade, on 
the average, with those skills intact. However, for students at the lower 
reaches of the class range, this gain would not be sufficient to bring their 
performance up to grade level. Even for students in the middle of the range, 
silent reading comprehension and listening scores are inadequate. 

Outcome results based on the pretest clas/if ications indicate that 
students in two of the classifications had significantly different achievement 
depending on the teacher they had. 

a. In Diagnostic Classification 2 (N « 25) there was a significant 
teacher effect (p <.05) for word recognition outcomes. Across the 
seven teachers who had students from that classification mean student 
gain was .60 years with a range of .18 to 1.17 years. For this same 
entering classification there was an additional teacher effect 
approaching significance (p <.10) for silent reading comprehension 
outcomes as well. Mean gain was .71 years; the range was 0 to 1.67 
years. 

b. In Diagnostic Classification 3 (N ■ 24) there was a statistically 
significant teacher effect (p <.05) for listening comprehension out- 
comes. The mean gain was .68 with a range across the seven teachers 
who had students from this classification of 0 to 2.0 years. 

Thus the teachers in this study were differentially effective for 
students with identical entering diagnostic classifications who could have 
been expected to show comparable achievement by year's end. 

Differential effectiveness for whole-class outcomes for all 10 teachers 
was also documented for word recognition (p <.05) and silent reading compre- 
hension (p <.01). Mean gain for word recognition was .49 years; the range was 
.22 to .80 years. Mean gain for silent reading comprehension was .37; the 
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range was -.16 to +1.25 years. Thus, regardless of the diagnostic charac- 
teristics of individual students, some teachers were more effective than 
others in improving student achievement on certain reading skills* 

In sum, the analysis procedure differentiated among students and 
teachers. It allowed for the reliable diagnostic classification of students 
based on their performance on specified reading tasks and for tracking pre- 
cisely achievement within those classifications. The procedure also was 
capable of documenting differential teacher effects on (a) achievement of 
students grouped within the same pretest classification and (b) achievement 
across students regardless of their individual diagnostic classifications. 

An effort was made to see whether special practices could be identified 
for those teachers whose students made above average gains in achievement. 
The accuracy of the data from the first interview was verified via classroom 
observations made by the research staff. Within the constraints of observing 
each teacher once, researchers concluded that an overall match existed between 
interview and observational data. The interview and observational data 
collected were insufficient as a basis for differentiating among the teacher^. 
The overall similarity of responses during the first interview masked any 
differences that undoubtedly existed among the various classrooms. However, 
the analysis of achievement data did permit us to show that specific teacher 
effects on achievement can be documented and that teachers can be rank ordered 
from highest to lowest according to the level of their students' achievement 
in specific reading skill areas. Thus the analytic procedures provide a basis 
upon which to select teachers for further observation and interview. 

Below are the responses to 10 questions about classroom practice asked 
during the first interview. They point to the presence of a small, shared, 
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repertoire of instructional practices in reading promoted, perhaps, by the 
district having recently imposed a uniform set of district-wide basal 
materials together with their associated objectives. 

1. Texts and materials used . Nine of the 10 teachers used the 
Houghton-Mifflin basal series mandated by the school district for 
grades 1-5. The accelerated class used the basal mandated for 
grades 6-8. Most teachers made periodic use of supplementary 
materials, such as library books and commercially published skills 
exercises. 

2. Students requiring the most instructional time . Teachers per- 
ceived themselves as giving more time and attention to low students 
than to other student groups. 

3. Feedback to students . The most commonly mentioned methods of 
feedback were written and verbal comments and sharing and displaying 
good work* 

4. T ypes of motivational activities . Most of the teachers reported 
the use of verbal praise, tangible rewards, and communicating their 
personal enthusiasm about reading. 

5. Overall choreography of the reading lesson . Seven of the 
teachers reported that they grouped their students. They either 
rotated the groups through a fixed series of daily activities or 
through a series of weekly activities, with each activity assigned 
to a different day. Whole-group instruction was also used in con- 
junction with the grouping. 

6. Criteria for grouping . Most of the teachers reported using some 
combination of the following criteria to determine group membership: 
students 1 placement at the end of the preceding year, basal place- 
ment test results, teacher judgment, reading teacher recommenda- 
tions, and standardized test scores. 

Teachers were asked to describe how th y typically conducted instruction 
in oral reading, word recognition, comprehension skills, and listening skills. 

7. Oral reading . Most teachers dealt with miscues by providing 
the student with the correct word; a few directed students to 
reread or to attempt to decode the word. Problems with inflection 
and fluency were typically addressed by teacher modeling. 

8. Word recognition skills . The three instructional strategies 
mentioned most often were using context to determine an unknown 
word, using the basal skill exercises, and teaching decoding via 
direct instruction. 

9. Comprehension skills . The most commonly used strategies were 
questioning and group discussion. 
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10. Listening skills . Three types of favored activities were use 
of commercially prepared tapes and kits, practice in listening to 
directions, and use of teacher-designed games. 



Student Interview Results 

Student responses to the interview questions posed on the pre- and 
posttest diagnostic batteries were analyzed to see if they could provide 
additional predictive data about student diagnostic classifications. A numbe 
of items were found to be significantly correlated with performance (p < .01) 
on one or more of the four diagnostic reading behaviors listed below. 
However, the usefulness of student interview data in contributing to the 
quality of diagnostic classification is uncertain. 



Student interview responses which correlated with test performance 
(* significant both for pre and posttests, WR = Word Recognition, 
OR « Oral Reading, SRC » Silent Reading Comprehension, and LC » 
Listening Comprehension): 

Performance (in specified diagnostic categories) was positively 
correlated with: 

* a) fewer retentions (WR, OR, LC); 

b) understanding science texts (WR); 

* c) using the dictionary for word meanings (WR) ; 

* d) characterizing good oral reading by inflection (LC) ; 

e) understanding the English text (WR); 

f) "figuring out" meaning of unknown word (SRC, LC); and 

g) having a plan for decoding words (LC). 

Performance (in specified diagnostic categories) was negatively 
correlated with: 

* a) seeing oneself as being a good reader (WR, OR, LC); 

* b) not wearing prescribed glasses (WR, OR); 

c) not reporting any strategy for decoding (LC); 

* d) not being able to characterize good oral reading (LC); and 
e) frequent absence in second grade (LC). 
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Summary and Discussion 



This study was the last In a programmatic series of research studies 
which began by investigating diagnostic reliability in reading, continued by 
developing a training program for improving reliability, and concluded by in- 
vestigating how diagnostic information could be used to improve student 
achievement in reading. We examined (1) teacher responses to receiving diag- 
nostic information about some of their students; (2) effects on student 
achievement of teachers receiving both diagnostic information and diagnostic 
training; (3) diagnostic classifications that emerged from student performance; 
and (4) differential teacher effectiveness and instructional practices. 

The fifth-grade teachers responded positively to the receipt of specific 
diagnostic information about their students, seeing in the information poten- 
tial for thinking about their own practice and for making changes in their 
reading instruction* 

There were no apparent effects on student achievement either for teachers 
receiving diagnostic information or for teachers receiving diagnostic training 
in addition to diagnostic information* 

Eighteen diagnostic classifications accounted for the test performance of 
all 169 students in the study; five classifications accounted for 60% of the 
students* The characteristics of the classifications together with their 
instructional implications were presented* Teachers* grouping practices did 
not correspond to the diagnostic characteristics we found for their students* 

Teachers were found to be differentially effective for students in three 
of the five major diagnostic classifications. In two diagnostic classifica- 
tions, ceacher effects for the entire class were also documented* Available 
interview and observational data were not sufficient for specifying instruc- 
tional practices associated with varying levels of achievement. 
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The system for capturing and interpreting reading-related data documented 
by the study has importance for different educational communities. For educa- 
tional researchers, the system provides a means of establishing, over time, an 
empirical basis for improving instruction and achievement in reading by (1) 
identifying instructionally meaningful diagnostic classifications based on 
student performance and (2) identifying teachers who appear to be especially 
effective in promoting student achievement in specified reading areas. 

For participating teachers, the system makes more accessible already 
existing student information. Individual teachers can use diagnostic and 
performance information to evaluate specific instructional decisions and 
strategies in the light of student achievement. 

The project findings suggest that developing this empirical body of in- 
formation will require a large sample of students and teachers in order to 
establish the most common diagnostic classifications for particular grade 
levels together with the expected outcomes for students in those classifica- 
tions. Further, teachers shown to be effective in promoting student achieve- 
ment in specified areas at the various grade levels must be interviewed and 
observed intensively so that those instructional practices associated with 
higher levels of student achievement can be made explicit. A data base that 
verifies instructional techniques of choice for students in specific diag- 
nostic classifications would serve to enlarge teachers* repertoires of rele- 
vant instructional activities, act as a clearinghouse for disseminating 
existing exemplary practice, and provide a common basis for discussion and 
exchange among teachers and between teachers and the research community. 

The strength of the system described in this study is that it, too, 
learns from experience. The data base that allows teachers to learn from 
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their own and their colleagues 1 experience would itself be altered by the 
continuous addition of teacher actions and their consequences. It is a system 
that would ultimately prescribe treatments of choice, but that would simul- 
taneously, because of its grounding in the observation of practice, be re- 
sponsive to variations of practice and associated outcomes. It is a system 
capable of both learning and teaching. 
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Name of school 
Name of teacher:. 
Nue of student:. 
Name of tester:^ 



THIS SHEET MUST BE REMOVED BEFORE DATA IS PROCESSED! 
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Inatruc tlona to Test Administrator for Total Test 

1* Please record today's date and the tine: 
Date : Time : 

2. When the cover sheet identifying the school, teacher, student 
and tester names is destroyed, the codes for this information 
should be inserted here. 

School code: 

Teacher code: 

Student code: 

Tester code: 

2- The following materials should be present for the Ustins 
session: 

a. The 'Test Materials Notebook* 

b. A stop watch for timing various parts of the testing 
session. 

c. A wristwatch or clock* 

d. One sheet of opaiue 8.5 X 11 inch paper. 

e. Twd I 2 2/4 pencils with eraser. 

M. DO NOT GIVE THE STUDENT HIS/HER TEST MATERIALS AT THIS TIME. 



Instructions to Test Administrator for Total Test 



Out cone studies in Reading 
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Introduction to Student 



After greeting the student and introducing yourself, orient 
•the student to the session. Be sure to cover the following 
points: 

1. that all the students in this class as well as in a number of 
other classes are being tested; 

2. that the purpose of the testing is to collect information on 
how students read; 

3« that we hope the information will be useful to teachers as 
they help students read better; 

1. that the session will last about half an hour. During most of 
that tine the student will be reading either out loud or to 
him/herself; 

5. that during the session you will be Jotting things down and 
using the stopwatch from tine to tine; 

6. let the student ask you any questions s/he night have. 



Introduction to Student 
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Standard Interview Porta 



1. Please record the clock tine. Tine: 

Read the following to the student: 

"I'M GOING TO ASK YOU SOME QUESTIONS. YOUR ANSWERS WILL KELP 
US BETTER UNDERSTAND HOW STUDENTS READ. 

Unless otherwise directed, check only one answer per 
question. If the student doesn*t respond, prompt with the 
listed options. 

FOR PRETEST, ASK QUESTIONS IN SECTIONS 2 AND 1. IF THIS IS 
NOT AN INITIAL TESTING, ASK QUESTIONS IN SECTIONS 3 AND 

2. FOR PRETEST SESSION 



a. What kind of reader do you think you are? 

very good (1) 

_j5<x>d (O.K.) (2) 

not very good (3) 

_bad 

no response or didn*t know (-1) 

other (-2) 



b. (Ask only if student indicates "not very good" or "bad*.) 
What grade level were you in when you first noticed that 
reading was a problem? 

1st grade d) 

2nd grade (2) 

3rd grade (3) 

_J»th grade (*) 
__5th grade (5) 

no response or didn't know (-1) 

other (-2) 

c. Have you been absent froa school a lot in past years? 
_yes (1) 

_no (2) 

so«e grades (3) 

no response or didn*t know (-1) 

other (.2) 



d. (Ask only if student indicated absences.) 

In what grade(a) were you absent a lot? (check all that 
apply) 

1st grade (yes=1 no=0) 

2nd grade (yessl no=0) 

3rd grade (yes=1 nosO) 

*th grade (yes=1 no=0) 
5th grade (yes=1 no=0) 



Standard Interview Fora 



^■MAILABLE 
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e. Have you ever repeated a grade? 
_jres (1) 
_no (2) 

no response or didn't know (-1) 

other (-2) 

00 TO SECTION *4 KOV! 



3. FOR POSTTEST SESSIONS* 



a* How often have you been absent from school this year? 

less than 1 week O) 

1 week - 3 weeks (2) 

more than 3 weeks (3) 

no response or didn't know (-1) 

_other (-2) 



b. How would you say your reading now compares with your 
reading at the beginning of the year 7 
_better (1) 

the sane (2) 

worse (3) 

no response or didn't know (-1) 

other (-2) 



00 ON TO SECTION U NOW! 



*l. COMPLETE THE QUESTIONS IN THIS SECTION FOR ALL 
ADMINISTRATIONS OF THE TEST BATTER!. 

a. When you read your school books, do you know most of the 
words ritfit away? 

yes (1) 

__no (2) 

no response or didn't know (-1) 

_other (-2) 



b. What do you do when you come to a word you can't read 
rigit away? (check all that apply) 

skip it (yesrl no=0) 

ask for help (yesrl no=0) 

use sound symbol association (e.g., sound it out) 

(yesrl no=0) 

use unspecified strategy (e.g., figure it out) (yes=1 

nozO) 

no response or didn't know (yes- 1 no-0) 

other (ves-1 no-0) 



Standard Interview Fora 

31 
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c When you listen to people read out loud, how can you tell 
if they're doing a good Job? (check all that apply) 
^accurate (e.g., they know all the words) (yes=1 no=0) 

inflection (e.g., they read smoothly) (yessl norO) 

speed (e.g., they read fast) (yesrl no=0) 

no response or didn't know (yes- 1 no-0) 

other (ves-1 no-0) 

d. Do you have any trouble seeing things clearly7 



no response or didn't know (-1) 

other (-2) 



e. Do you have glasses for reading? 
_yes (1) 
_no (2) 

no response or didn't know (-1) 

other (-2) 



f. (Ask only if the student has glasses.) 
Do you wear the glasses when you read? 
—yea CD 

sometimes (2) 

_no (3) 

no response or didn't know (-1) 

other (-2) 



g. Do your eyes bother you after you've been reading for 15 
or 20 minutes7 

sometimes (2) 

_no (3) 

no response or didn't know (-1) 

other (-2) 



h. Do you hc/e trouble hearing your teachers while you're 
trying to listen to then? 
— yea O) 

sometimes (2) 

_no (3) 

no response or didn't know (-1) 

other (-2) 



Do you have any trouble understanding what you read in 
your: 

i. science books? 
—>es (1) 

sometimes (2) 

_no (3) 

no response or didn't know (-1) 

__other (-2) 



Standard Interview Form 
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J. English books? 

—yea < 1 > 

sone times (2) 

_no (3) 

no response or didn't know (-1) 

_other (-2) 



k. social studies books? 
_yes (1) 

sometimes (2) 
__no (3) 

no response or didn't know (-1) 

_other (-2) 

1* math books? 

sometimes (2) 

_no (3) 

no response or didn't know (-1) 

__other (-2) 



m. If your teacher asked you to sound out a word, how would 
you do it? (check all that apply) 

divide it up (syllabicate) (yes-i no»u) 

sound it out(yes*l no-0) 

put it all together (blend/adjust) (yes-1 no-0) 

other (yes-1 no«0) 

no response or didn't know (yes-1 no-0) 

n. What do you do when you come to a word you can read but 
don't know its meaning? (check all that apply) 

skip it (yes=1 no=0) 

ask someone (yes=1 no=0) 

look in the dictionary (yes=1 no=0) 

try to figure it out (yes=1 no=0) 

no response or didn't know (ve*-I no-0) 

other (yes*l no«0) 



Word Recognition Measure 



1. Please record the clock time. Time: 



2. Give the student his/her "Test Materials Booklet". Turn to 
the SORT. 

3. Read the following directions to the child: 

"HERB ARE SOKE LISTS OF WORDS. I'LL ASK YOU TO READ SOME Of 
THESE LISTS. IF YOU DON'T KHOW A WORD RIGHT AWAY, TRY AND 
SOUND IT OUT. WE'RE GOING TO START WITH THE VERY FIRST LIST.* 

*l. For each word, record student performance as follows: 

a. If the student reads the word correctly with no 
hesitation, check the * Correct 1 column. 

b. If the student does not correctly call the word within a 
slow count of five, check the 'Incorrect* column. An 
incorrectly accented word is considered incorrect. If the 
student does not respond, check the 'Incorrect' column. 

c. If you see or hear a student attempting to decode a word, 
check the 'Mediated 1 column irrespective of whether the 
word is pronounced correctly or incorrectly. 

d. After a slow count of five, ask the student to continue 
with the next word. 

5. Let the student proceed through the lists in order as long as 
six or core words are marked as 'Correct'. 



REMEMBER: THE CRITERION FOR CONTINUING IS 6 OR MORE WORDS 
READ CORRECTLY - INSTANT OR MEDIATED. 



Standard Interview Form Word Recognition Measure 



BEST cop y m, LABLE 



o 33 34 

ERIC 



Outcoae Studies in Reading 



8 



Word Correct Incorrect Mediated 

1. Jump 

2. want 

3. look 

4. play 

5. coae 

6. is _ 

7. can 

8. one 

9. see 

10. ball _ _ 

11. little _ 

12. three 

13. baby 

W. run 

15. down ____ 

16. Bother 

17. here 

18. up _ 

19. help 

20. cake 
TOTALS 




Word Recognition Measure 
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Word 



Correct Incorrect Mediated 



21. very 

22. „nder 

23. hill 
21. road 

25. dark 

26. horse 

27. came 

28. puppy 

29. was 

30. bump 

31. with 

32. ride 

33. along 
3*». friends 

35. first 

36. food 

37. wish 

38. what 

39. basket 
10. live 
TOTAU 



Word Recognition Measure 
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Word Correct Incorrect Mediated 

41. better _ 

42. hide _ 

43. large 

44. field 

45. around 

46. suddenly 

47. forest 

48. river 

49. sheep 

50. station 

51. breakfast 

52. across 

53* heavy 

54. lunch 

55. happen 

56. grass 

57. gane 

58. fanner 

59. stars 

60. hope _ 

TOTALS 



Word Recognition Measure 
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Word Correct Incorrect Mediated 

61. hunger 

62. evening 

63. bench 

64. excuse 

65. grove 

66. desire 

67. perform ^ 

68. destroy 

69. delicious 

70. understood _____ 

71. safe 

72. against 

73. timid 

74. ocean 

75. damp 

76. streao 

77. eopty 

78. smash 

79. stone 

80. reward _____ 
TOTALS _____ 



Word Recognition Measure 



sesr copy 
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Word Correct Incorrect Mediated 



81, claincd 

82. forehead 
83* serious 

84. speechless 

85. distant 

86. dainty 

87. *nger 

88. courage 

89. sltKber 

90. COSitOi 

91. harness 

92. flakes 

93. derelop 
9 1 *. racant 

95. price 

96. 'uture 

97. ippearance 

98. promptly 

99. region 

100. silence 

tota:^ 
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Wonl Correct Incorrect Mediated 

101. dignity 

102. generally 

103. gracious 

104. applause 

105. define 

106. cushion 

107. Jungle 

108. haze 

109. profitable 

110. tailor _ _ 

111. fragrant 

112. obedient 

113. marriage 

114. presence 

115. terraco 

116. interfere 

117. aabition 

118. ner chant 

119. custoa ____ 

120. extended ____ 

TOTALS 



Word Recognition Measure 



Outcome Studies in Reading 



14 



Outcome Studies in Reading 



15 



Word Correct Incorrect Mediated 

• 4 t 

121. tyesendous 

122. dungeon 

123. custoaary 

124 . abundant 

125. aalicious 

126. imaginary 

127. excellence 

128. consequently 

129. installed 

130. responsible 

131. yearning _____ _____ 

132. liquid 

133. ned i cine _ 

134. spectacular 

135. rebellion 

136. cotapliaents 

137. infected 

138. inventory _____ _____ _____ 

139. iaportance _____ ______ 

140. detained 

TOTALS 



Word Correct Incorrect Mediated 

141. renini sconce 

142. architecture 

1*13. envircnaent 

144. counterfeit 

145. pensive 

146. trcaor 

1*17. intricate 

148. adoinister 

149. continuously 

150. attentively 

151. industrious 

152. contemporary 

153. approximate 

15*1. standardize 

155. coaplexion 

156. exhausted 

157. nallgnant __ 

158. coapassionate _ ______ 

159. crisis 

160. society _ ____ 

TOTALS 



Word Recognition Measure Word Recognition Measure 
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Word Correct Incorrect Mediated 

161. nucleus 

162. proportional 

163. articulate 

164. remarkably 

165. nonchalant 

166. memorandum 

167. deprecate 

168. contrasting 

169. whimsical _ 

170. evident _ _ 

171. prairies 

172. twilitfit _ 

173. intangible 

174. supplement 

175. formula ted 

176. exuberant 

177. antique 

178. induceaent 

179. irrelevance 

180. grotesque 

TOTALS 
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Word Correot Incorrect Mediated 

181. sojourn 

182. affable 

183. auspicious 

18*1. enamoured 

185. futility 

186. primordial 

187. inadequacy 

188. chastisement 

189. envisage 

190. pandemonium 

191. panorama 

192. compressible 

193. gustatory _ 

194. traverse _ 

195. simultaneous _ 

196. excruciating 

197. contraband 

198. decider _ 

199. scrupulous 

200. facsimile 

TOTALS _ _ 



Word Recognition Measure 
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Airplane pilots have many important jobs. They f Jy 

passengers, freirfit, and mail from one city to another. Sometimes 

they nake dangerous rescues in lai>4 wid sea accidents, and drop 

food where people or herds are starving. They bring strange 

aniaals from dense Jungles to our zoos. They also serve as 

traffic police and spot speeding cars on hlgiways. 

Stopwatch time: Inflection YES WO 

Fluency YES NO 

G5 

Hundreds of years ago, most of Europe was a very poor 

region. But China, a large country in eastern Asia, had many of 

the coaforts of a rich, civilized nation. Only a few people froa 

Europe had visited this distant region. One was the fanous Marco 

Polo. He learned soae of the languages that were spoken in China 

and served its great ruler for oany years. 

Stopwatch tiae: Inflection YES HO 

Fluency YES NO 

G6 

The eager spectators who had cheered the plucky Warriors 

through eirfit hard-fougjt innings were silent. Only a run was 

required to defeat the ouch feared chaapions, who had previously 

defeated all opponents. The spectators had earlier criticized the 

uapire severely. Now their faces were tense with exciteaent as 

the players took their positions. 

Stopwatch time: Inflection YES NO 

Fluency YES NO 
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The oil industry has been greatly increased by recent 

advances in science. Geologists have discovered new ways of 

locating veins of oil-producing rock. Problems of gusher control 

have been solved. Very effective also are newer methods of 

refining crude oil which have resulted in a higier ratio of 

quality fuel oil froa a given voluae of crude oil. 

Stopwatch tiae: Inflection YES NO 

Fluency YES NO 

G10 

In response to the iapulse of habit Joseph rose and spoke as 

in foraer days. He spoke vigorously, continuously, and 

persuasively while the others listered attentively but in grie 

and conteaptuous silence. Finally exhausted, Joseph hesitated for 

a aoaent; as often happens in such circuastances he becaae 

confused and was unable to resume speaking. 

Stopwatch tiae: Inflection YES HO 

Fluency YES NO 

G12 

Many of the hypotheses about physical phenomena formulated 

by early philosophers were inconsistent and in soet cases could 

not be universally applied. In order to develop accurate 

principles very capable physicists, mathematicians, and 

statisticians had to cooperate wholeheartedly over long periods 

of tiae to verify nuaerous basic facts and assumptions. 

Stopwatch tiae: Inflection YES NO 

Fluency YES NO 



Gray Oral Reading Test 
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G* 

Airplane pilots have many important Jobs. They f )y 

passengers, frei£>t, and mail from one city to another. Sometimes 

they make dangerous rescues in lane: tnd sea accidents, and drop 

food where people or herds are starving. They bring strange 

animals from dense Jungles to our zoos. They also serve as 

traffic police and spot speeding cars on highways. 

Stopwatch time: Inflection YES NO 

Fluency YES HO 

G5 

Hundreds of years ago, most of Europe was a very poor 

region. But China, a large country in eastern Asia, had many of 

the comforts of a rich, civilized nation. Only a few people from 

Europe had visited this distant region. One was the famous Marco 

Polo. He learned some of the languages that were spoken in China 

and served its great ruler for many years. 

Stopwatch time: Inflection YES HO 

Fluency YES NO 

G6 

The eager spectators who had cheered the plucky Warriors 

through ei#)t hard-fou#)t innings were silent. Only a run was 

required to defeat the much feared champions, who had previously 

defeated all opponents. The spectators had earlier criticized the 

umpire severely. Now their faces were tense with excitement as 

the players took their positions. 

Stopwatch time: Inflection YES NO 

Fluency YES NO 




Gray Oral Reading Test 
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OS 

The oil Industry has been greatly increased by recent 

advances in science. Geologists have discovered new ways of 

locating veins of oil-producing rock. Problems of gusher control 

have been solved. Very effective also are newer methods of 

refining crude oil which have resulted in a higher ratio of 

quality fuel oil from a given volume of crude oil. 

Stopwatch time: Inflection YES NO 

Fluency YES HO 

G10 

In response to the impulse of habit Joseph rose and spoke as 

in former days. He spoke vigorously, continuously, and 

persuasively while the others listered attentively but in grim 

and contemptuous silence. Finally exhausted, Joseph hesitated for 

a moment; as often happens in such circumstances he became 

confused and was unable to resume speaking. 

Stopwatch time: Inflection YES HO 

Fluency YES MO 

G12 

Many of the hypotheses about physical phenomena formulated 

by early philosophers were inconaistent and in most cases could 

not be universally applied. In order to devalop accurate 

principles very capable physicists, mathematicians, and 

statisticians had to cooperate wholeheartedly over long periods 

of time to verify numerous basic facts and assumptions. 

Stopwatch time: Inflection YES NO 

Fluency YES NO 
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1. Please record the clock time. Tine: 



2. Turn to the Silent Reading Paragraphs. 

3. Read the following instructions to the student: 

"YOU ARE GOING TO BE READING SOKE SELECTIONS SILENTLY ONE AT 
A TIME. I'LL BE ASKING IOU ABOUT EACH SELECTION AFTER YOU 
READ IT. I'LL TELL YOU WHEN TO START. PLEASE TELL ME WHEN YOU 
HAVE FINISHED READING." 

*. For each selection do the following: 

a. RESET THE STOPWATCH TO ZERO. 

b. Tell the student to begin. Then, START THE STOPWATCH. 

c. When the student has finished the selection, stop the 
stopwatch, record the time and cover th* selection with 
the 8.5 x 11 opaque paper. 

5. For each selection conduct the comprehension procedure as 
follows: 



a. Say to the student, "TELL KB WHAT WAS IN THE SELECTION, 
STARTING AT THE BEGINNING." Number all memories in order 
of recall. Ignore unrelated responses. 

b. The criterion for stopping is when the student remembers 
fewer than 40* of the memories. The number of recalls 
required to continue to the next paragraph are given with 
each selection. 

c. Start the student at paragraph 3. If the student is 
stopped at paragraph 3, then try 2, etc. If the student 
is not stopped at paragraph 3, go on to A, etc. 



REMEMBER: THE CRITERION FOR CONTINUING IS WHEN RECALL 
REACHES OR EXCEEDS THE NUMBER OF MEMORIES GIVEN WITH EACH 
SELECTION. 
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1. 1: Criterion is: 6 memories. Time: COVER SELECTION 1 



2. Tell me what the story was about, starting at the beginning. 
A hen had 

six yellow chicks. 

One morning 

she took them for a walk. 

They looked for 

something to eat. 

They found some seeds 

and sand. 

A dog came 

to play with them. 

The hen 

did not like the dog. 

_JShe flew at the dog 
and made him run away. 



1. 2: Criterion is: 8 memories. Time: COVER SELECTION! 



2. Tell me what the story was about, starting at the beginning. 

Two children 

got off the bus. 

There was nobody 

at the bus stop 

to meet them. 

__So they sat down 
__to wait* 

A few minutes later 

an old blue car 

came around the corner. 

The children's mother 

_waa in it. 
She got out and 

said she was late because 

she couldn't get the car started 

and she had to fix it. 

They all 

got into the car 

and drove home. 
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1. 3: Criterion is: 10 memories. Tine: COVER SELECTION! 



2. Tell no what the story was about, starting at the beginning. 
Last week 

a boy and girl from our school 

had a real adventure. 

_They were going past the bank 

on their way hone 

for lunch 

when two men ran out 

with a bag of money . 

The men had a car waitirig for thea 

and drove away 

very fast. 

Wien the police came, 

the children told them 

what color the car was 

_and how big it was. 

They could even tell the police 

__that one man was short and 

the other one was tall. 

Because the children were 

in the ritfit place 

that day, 

the men were soon caught and 

the money was returned. 
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1. 4: Criterion is: 11 memories. Time: COVER SELECTION! 



2. Tell ae whet the story was about, starting at the beginning. 

Nearly two hundred 

years ago 

in France, 

the first man 

went up in a balloon. 

The balloon was made of 

paper 

covered with strips of cloth 

to make it strong. 

A huge basket 

was attached underneath 

to carry people, 

and a long rope 

anchored it 

to the ground. 

On this first trip, 

. the man was in the air 

alone for several minutes. 

Later he took a friend 

up in the balloon with him. 

That time, they rose 

core than five hundred feet 

and stayed up 

for half an hour. 

They came down 

several miles 

from vhe^e they started. 
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1. 5: Criterion is: 12 memories. Time: COVER SELECTION! 



2. Tell ae what the story was about, starting at the beginning. 

At one time, imaense herds of buffalo 

_roamed the Great Central Plains 

of North America. 

Indian tribes 

like the Blackfoot and the Cree 

called theaselves "People of the Buffalo" 

»nd followed these huge animals, 

killing them with bowa and arrows 

for the food and skins they needed. 

They hunted on horseback 

and moved their shelters 

__with the herds. 

About one hundred years ago, 

this source of food 

was lost to the Plains Indians. 

?he buffalo began to be killed 

in large numbers, 

not Just for needed food, 

but for buffalo robes. 

J* 10 robes brought such high prices 

that some hunters pursued the animals 

_with rifles, 

traveling by train 

to where they grazed. 

In a short time 

__the buffalo had all been killed, 

_and the Indians were left 

without a basic source of food. 
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1. 6: Criterion is: 14 memories. Time: OOVBR SELECTION! 



2. Tell me what the story was about, starting at the beginning. 
No one knows 

exactly how or where ice hockey started. 

However, field hockey was played 

in England 

_more than one hundred fifty years ago, 

and soae pucks and sticks 
_Jhave been found 

that show 

that ice hockey 

was played 

in Canada 

_J>y British soldiers 
_during the 1850 »s. 
_By 1890, 

a hockey association had been formed 

in Canada 

to administer 

and develop the game. 
__Shortly after, hockey was introduced 

into the United States 

at two universities 

and the game was soon very popular. 

_At first 

the game was played 

only for fun, 

but people were willing 

_to pay 

to watch exciting games. 

So professional teams were set up, 

_and the National Hockey Association formed. 

That association later became 

the National Hockey League (NHL) • 

By the max of the 1970* s, 

there were nearly twenty teams 

playing in that league. 
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1. 7: Criterion is: 12 memories. Time: COVER 

SELECTION! 

2. Tell us what the story was about, starting at the beginning. 
Every four years 

athletes 

from all over the world 

con pete 

in a festival of sports. 

These great contests 

are called the Olyapio Gaaes, 

since they were first conducted 

at Mount Olyapus 

by the ancient Greeks. 

They were part of a religious celebration 

in honor of Zeun, 

father of the Greek gods. 

They were discontinued long ago 

and were revived 
__only in 1896 
by aen 

interested in continuing the custoa. 

The first 

of the modern gaaes 

were staged appropriately 

in Athens. 

Since that tiae 

they have been held 

in various countries. 

They stimulate world friendship 

and help to maintain interest 

in physical perfection. 
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1. 8: criterion is: 13 aeaories. Tiae: COVER 

SELECTION ! 

2. Tell oe what the story 'ras about, starting at the beginning . 
One of the nost difficult 

political 

and economic 
problems 

of our Federal government 

is to prevent 

the developaent 

throughout the country, 

of certain commercial interests 

at the expense of others. 

One very persistent attack 

has been against 

the development 

of large corporations. 

DeWitt 

in his study of the history 

of this struggle 

states 

that it has three distinct objectives. 

First, to find some satisfactory means 

to control 

and regulate 

the activities of large business corporations; 

second, to resist 

the tendency 

of corporations 

to exploit 

natural resources 

for their own benefit; 

third, to control tariffs 

which favor trusts 

and monopolies. 
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Listening Comprehension Measure 



1. v» Please record the clock tioe. Time: 

2. Take the test booklet away fron the student. You will be 
reading the Listening Comprehension Paragraphs to the 
student. 

3. Read the following instructions to the student: 

"YOU ARE GOING TO BE LISTENING TO SOS SELECTIONS. I'LL BE 
ASKING YOU ABOUT EACH SELECTION AFTER YOU'VE LISTENED TO IT. 

4. Read at a moderate pace using nonsal inflection. 

5. For each selection conduct the coa prehension procedure as 
follows: 

a. Say to the student, "TELL ME WHAT HAS IN THE SELECTION, 
STARTING AT THE BEGINNING." Number all memories in order 
of recall. Ignore ^related responses. 

b. The criterion for stopping is when the student remembers 
fewer than 40^ of the aeoories. The number of recalls 
required to continue to the next paragraph are given with 
each selection. 

c. Start the student at paragraph 3. If the student is 
stopped at paragraph 3, then try 2, etc. If the student 
is not stopped at paragraph 3, go on to M, etc. 



REMEMBER: THE CRITERION FOR CONTINUING IS WHEN RECALL 
REACHES OR EXCEEDS THE NUMBER OF MEMORIES GIVEN WITH EACH 
SELECTION. SPECIFIC NUMBERS ARE GIVEN WITH EACH 
SELECTION. 
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1. LCI: Criterion is 7 meaories 

2. Tell me what the story was about, starting at the beginning. 
A boy 

had a big gray cat. 

He was going to give her 

2^ae milk. 

The cat 

did not come 

when he called. 

He saw her 

up in a tree. 

She was looking down 

at a big dog. 

The boy 

sent the dog away. 

Then the cat 

Jumped down. 

She came 

for some milk. 



1- LC2: Criterion is 7 memories 

2. Tell me what the etory was about, starting at the beginning. 

Jan woke up 

and ran down 

to breakfast. 

It was her birthday. 

She saw 

a big basket 

on her chair. 

Something was moving 

in the basket. 

She took off the cover 

and a little brown dog 

Jumped out. 

The dog 

started to bark 

because it was glad 

to get out. 
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1. LC3: Criterion is 8 memories 

2. Tell ne what the story was about, starting at the beginning. 
A boy 

was hurt 

on our street 

yesterday. 

He had been playing ball. 

He was riding 

hi3 bicycle 

_away from the ball field 

when a car 

pane down the road. 

He did not see the car coming 

because he was looking back 

at the children 

who were still playing ball. 

The car was going slowly. 

_It hit the boy 

but did not run over him. 

His arm was hurt 

and his bicycle was bent. 



Listening Comprehension Measure 



Outcome Studies in Reading 



33 



1. LCS: Criterion is 11 memories 

2. Tell me what the story was about, starting at the ueginning. 
The first railroad engines 

were pulled 

by horses 

along rails. 

Later 

steam engines were employed 

to pull trains, 

but these first engines 

could not travel very rapidly 

and often broke down. 

Once 3ome people 

had a contest 

between a train 

drawn by a horse 

and a train 

powered by an engine. 

At first the horse was ahead 

because the engine started slowly. 

Then the engine 

passed the horse 

and stayed ahead for a while. 

However, before the end of the race, 

the engine broke down 

and the horse wen. 

People who loved horses 

were pleased about the results 

of the contest. 
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1. LC5: Criterion is 14 memories 

2. ^ell Be what the story was about, starting at the beginning. 

S one tines we have the idea 

from television 

that the only heroes 

__in the "Wild West" 

were the cowboys 

and Indians. 

Really, the heroes of the West 

were the sen and women 

who case from far away 

crossing the endless prairie 

in covered wagons 

and floating their belongings 

across rivers. 

Many settlers 

traveled hundreds of miles 

before they reached land 

that looked promising to then 

and they could build 

some kind of shelter. 

They could claia 

only as ouch land 

as they could work 

for a certain number of year*?. 

Many died 

of cold, hunger, and overwork, 

but others survived 

and opened the broad western plains. 

These people 

cane with nothing 

but what they could carry, 

and a willingness to work. 

We should be grateful 

that they opened the West 

for us. 
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1. LC6: Criterion is 13 memories 

2. Tell se what the story was about, starting at the beginning. 
Baseball has often been called 

the national sport of the United States. 

It developed from games 

known in England 

as "rounders" 

and "town ball" 

and was played 

in U.S. colleges 

as early as 1825. 

Baseball's popularity 

has constantly increased 

because the rules are easily understood 

and the players require 

only simple equipment. 

War, as it happens, 

has been responsible 

for the growth of the pastime. 

Many men 

learned it first in camps 

during the American Civil War 

and started teams 

after they returned home. 

Both World Wars 

extended it further, 

for wherever U.S. soldiers were stationed 

in foreign oountries, 

they created 

an interest in baseball 

that remained after they left. 

In Japan, for example, 

baseball may now be as popular 

as it is in the United States. 
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1* LC7: Criterion is U memories. 

2. Tell ce what the story wis about, starting at the beginning. 
—--Failure to plan 

for suitable defense 

under irregular conditions 

of warfare 

accounted for defeat 

in the first war 

waged by the United States. 

——An army 

- — of two thousand een 

under General Arthur St. Clair 

marched northward 

from Cincinnati 

to punish Indians 

who had broken treaty provisions. 

They neglected 

to guard against 

— — unexpected assault 

and found themselves defenseless 

when hostile Indians 

suddenly attacked them 

in the forest. 

Firearms 

gave little protection 

against an eneay 

in an bush. 

After a futile attempt 

at defense, 

— -St. Clair ordered his oen 

to retreat. 

Only fifty escaped 

uninjured. 

President Washington 

felt very bitter 

about St. Clair's carelessness 

in the country's first military campaign. 
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1. LC3: Criterion is 12 memories 

2. Tell me what the story is about, starting at the beginning. 
Railroad communication 

developed rapidly 

Just after the civil war. 

Between 1865 and l873t 

thirty-five thousand 

miles of track were laid. 

This doubled the distance 

— —people could travel 
— — by railroad. 

Some of the new roads 

— —connected important cities 

— —and some extended westward beyond populated regions. 

Congrest 

favored this sudden 

— —development 

by granting land to companies 

— —interested in furthering 

the expansion. 

Grants included territory 

lying within twenty miles 

— —of the proposed roadbed. 

Alternate sections 

were allotted to the railroad. 

—Those in between were 

reserved for homesteaders. 

The sale of sections of land 

owned by the railroad 

was made easier 

through this checkerboard 

— -arrangeaont . 
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Test Administrator Debrl-flng Form 



1. ■ 

DO NOT DISMISS THE STUDENT YET. 



Examine the entire test booklet to sake sure that all tests 
are completed. Check then off. 
Interview 

Word Recognition (SORT) 

Oral Reading (Gray) 

Silent Reading (Durrell) 

Listening Comprehension (Durrell; 

If all are done, DISMISS the student. 

2. PLEASE RECORD THE CLOCK TIME! Time: 

Please answer the following questions concerning the j'ist 
coople ,ed testing session. 

3. What was the student's overall afreet? (Check all that 
apply. ) 

Animated 

Flat 

Sullen 

Hostile 

Passive 

Attentive 

Fidgety 

Nervous 

*J. What was the student's mean length or utterance? (Check all 
that apply.) 

Multiple Sentences 

Sentence 

Phrase 

One word 

5. What was the student's general sophistication of language? 
(Check all that apply.) 

High 

Average 

Low 

6. What was the student's sex? 
Male 

Female 

7. While reading silently and independently, what were the 
student's characteristics? (Check all that apply.) 
Squirmed 

Was attentive 

Gave up 

Asked for assistance 

Subvocalized 
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Tracked sentences with finger 

Raced througi material 

Other. Specify; 



8. Did the student appear to have any difficulty hearing? 
No 

Yes. Specify: 



9. Did the student appear to have any difficulty with the visual 
aspects of reading? 

Yes (Rubbing eyes, watery eyes, keeping text clo?e to 

eyes) 
No 

10. Enter any other comments about the session that have not been 
recorded elsewhere. 
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Appendix B: Interview 1 



Interview ffl 

Teacher 

Date 



Interview # 

School 

Grade 



1. How is the reading program set up in your school (self contained, joplin, 
reading specialist, etc.)? 



2. What time of the day do you schedule formal reading instruction? 

3. How much time do you allocate for formal reading instruction i.ach day? 
U. What reading tests and associated materials do you uce? 

5. Is any portion of your reading program mandated? If so, please describe. 



6. Do you operate under any other constraints which you feel affect your 
reading program? 



7. Overall, what kind of year are you having in reading? (What leads you 
to feel that way?) 



Describe the characteristics of the most common types of readers among 
your students. (This does not refer to your existing reading groups.) 



a) Considering the types of readers you've just described, which type 
of student do you think you are most successful with? 



b) Considering the type.? of readers you've described, which type of 
student do you think vou are least successful with? 



c) Which kinds of students require the most planning/preparation time? 
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d) Which kinds of students require the most instructional time? 

e) How do you let your students know about the quality of their work 
in reading? 

f) How do you try to motivate your students in reading? 

g) How do you bring a student up to date in reading when s/he has missed: 

1) one or two days of school? 

2) more than a week of school? 

h) In general, how would you rate the reading ability of your typical 
incoming students in September? 



9. Describe the choreography of a typical reading session* 

a) How many reading groups do you have and what criteria do you use to 
form the groups? 



b) How big is each group and what considerations determine its size? 

c) About how many minutes a day do you typically schedule to spend yith 
each group personally? (How close do you actually come?) 



d) How do you reevaluate group membership? 

e) Do you alter your instruction according to the group? 

f) Do you alter your instruction according to the time of year? 

g) Do you use your materials/resources differentially for each group? 
(How do you use them?) 

h) What is your major goal in reading for each of your groups? 
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10. How much of the textbook do you try to cover per week for each group? 
Per semester? Per /ear? (Additional materials beyond completion of 
text?) 



11, Describe how you conduct a typical oral reading session for each group* 
(How do you deal with miscalls and inflection?) 



12. How do you deal with acquisition of word recognition and analysis skills 
for each group? 



13. How do you deal with acquisition of comprehension skills for each group? 



14. How do you deal with acquisition of listening skills in general? 
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Appendix C: Interview 2 

Interview #2 

Teacher 

Date 



Interviewer 



School 



Grade 



1 # When we last talked together you explained your instructional practices 
in a number of areas (give laminated card with categories listed on it): 

Grouping (if any) 

Feedback and motivation 

Time allotted for the reading period 

Time allotted for each of the groups 

Materials used 

Instructional practices in: oral reading, word recognition 
and analysis, reading comprehension and listening comprehension 

Has anything changed since then in these areas? 

Have there been any other changes that have affected your reading 
program apart from these areas? 

(Take transcript of Interview One along. Prompt only if they say they can't 
remember what they said in Interview One.) 

2. Overall, what kind of year have you had in reading? 

3. You remember that you received information about how your students per- 
formed on the diagnostic battery for the pretest. Here are the charts we 
used. (Quickly go through each blank laminated chart and point out how 
each axis is labeled.) Each student was placed in a cell depending on 
how s/he did on the pretest. Later on, we sent you the pretest results 
expressed as average performance for your whole class as well as for each 
reading group (assuming you had groups). Please give me your thoughts 
about this pretest information. Free response. Prompt if following not 
covered in the response: 
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Useful, how? / not useful, why? 
Interesting? 
Format preference? 
Hard to understand? 



What was missing? 

4. Please give me your thoughts about the two sessions in which you 

diagnosed a simulated case of reading difficulty. Prompt if following 
not covered: 

Useful? / not useful, why? 



Interesting? 

Did it influence the way you interpreted the pretest information? 



After the posttesting is completed we will be sending you (at home) two 
sets of information* The first set will be formatted identically to that of 
the pretest (i.e., charts and class/group averages). The second set will show 
changes in performance from pretest to posttest. Would you be willing to let 
us arrange one last interview after you have had a chance to react to these 
results? You will of course receive a modest honorarium for your time. 
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Appendix D: Interview 3 
INTERVIEW #3: SEPT-OCT. 1984 



During this past summer you received posttest and achievement scores for last 
years' students in four reading diagnostic areas: word recognition, oral 
reading, silent reading comprehension and listening comprehension. 

1. Did the results support your own observations and judgments about 
your students achievement? Explain. 

2. Did the results contradict your observations /judgments in any way? 
Explain* 

3. Did you use the information in any way when planning for this year? 



4» What is your opinion of the usefulness of the posttest and achievement 
information you received? 



5. Did diagnosing the simulated cases twice have any influence on how you 
interpreted or used the post test/outcome scores? Explain, 



*6. Did the two diagnostic training sessions have any influence on how 
you interpreted or used the pos ttes t/ou tcome information? Explain. 



7. Could we have provided you with anything else that you feel would hfive 
had an influence on your instructional decisions and actions? 



* This question was asked only of those five teachers who received 
diagnostic training. 
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